Towards Lifelong Reasoning with Sparse and Compressive Memory Systems

Humans have a remarkable ability to remember information over long time horizons. When reading a book, we build up a compressed representation of the past narrative, such as the characters and events that have built up the story so far. We can do this even if they are separated by thousands of words from the current text, or long stretches of time between readings. During our life, we build up and retain memories that tell us where we live, what we have experienced, and who we are. Adding memory to artificial neural networks has been transformative in machine learning, allowing models to extract structure from temporal data, and more accurately model the future. However the capacity for long-range reasoning in current memory-augmented neural networks is considerably limited, in comparison to humans, despite the access to powerful modern computers. This thesis explores two prominent approaches towards scaling artificial memories to lifelong capacity: sparse access and compressive memory structures. With sparse access, the inspection, retrieval, and updating of only a very small subset of pertinent memory is considered. It is found that sparse memory access is beneficial for learning, allowing for improved data-efficiency and improved generalisation. From a computational perspective - sparsity allows scaling to memories with millions of entities on a simple CPU-based machine. It is shown that memory systems that compress the past to a smaller set of representations reduce redundancy and can speed up the learning of rare classes and improve upon classical data-structures in database systems. Compressive memory architectures are also devised for sequence prediction tasks and are observed to significantly increase the state-of-the-art in modelling natural language

Persistent starspot signals on M dwarfs: multi-wavelength Doppler observations with the Habitable-zone Planet Finder and Keck/HIRES

Young, rapidly-rotating M dwarfs exhibit prominent starspots, which create quasiperiodic signals in their photometric and Doppler spectroscopic measurements. The periodic Doppler signals can mimic radial velocity (RV) changes expected from orbiting exoplanets. Exoplanets can be distinguished from activity-induced false positives by the chromaticity and long-term incoherence of starspot signals, but these qualities are poorly constrained for fully-convective M stars. Coherent photometric starspot signals on M dwarfs may persist for hundreds of rotations, and the wavelength dependence of starspot RV signals may not be consistent between stars due to differences in their magnetic fields and active regions. We obtained precise multi-wavelength RVs of four rapidly-rotating M dwarfs (AD Leo, G 227-22, GJ 1245B, GJ 3959) using the near-infrared (NIR) Habitable-zone Planet Finder, and the optical Keck/HIRES spectrometer. Our RVs are complemented by photometry from Kepler, TESS, and the Las Cumbres Observatory (LCO) network of telescopes. We found that all four stars exhibit large spot-induced Doppler signals at their rotation periods, and investigated the longevity and optical-to-NIR chromaticity for these signals. The phase curves remain coherent much longer than is typical for Sunlike stars. Their chromaticity varies, and one star (GJ 3959) exhibits optical and NIR RV modulation consistent in both phase and amplitude. In general, though, we find that the NIR amplitudes are lower than their optical counterparts. We conclude that starspot modulation for rapidly-rotating M stars frequently remains coherent for hundreds of stellar rotations, and gives rise to Doppler signals that, due to this coherence, may be mistaken for exoplanets.Comment: Accepted for publication in the Astrophysical Journa

Persistent Starspot Signals on M Dwarfs: Multiwavelength Doppler Observations with the Habitable-zone Planet Finder and Keck/HIRES

Young, rapidly rotating M dwarfs exhibit prominent starspots, which create quasiperiodic signals in their photometric and Doppler spectroscopic measurements. The periodic Doppler signals can mimic radial velocity (RV) changes expected from orbiting exoplanets. Exoplanets can be distinguished from activity-induced false positives by the chromaticity and long-term incoherence of starspot signals, but these qualities are poorly constrained for fully convective M stars. Coherent photometric starspot signals on M dwarfs may persist for hundreds of rotations, and the wavelength dependence of starspot RV signals may not be consistent between stars due to differences in their magnetic fields and active regions. We obtained precise multiwavelength RVs of four rapidly rotating M dwarfs (AD Leo, G227-22, GJ 1245B, GJ 3959) using the near-infrared (NIR) Habitable-zone Planet Finder and the optical Keck/HIRES spectrometer. Our RVs are complemented by photometry from Kepler, TESS, and the Las Cumbres Observatory network of telescopes. We found that all four stars exhibit large spot-induced Doppler signals at their rotation periods, and investigated the longevity and optical-to-NIR chromaticity for these signals. The phase curves remain coherent much longer than is typical for Sunlike stars. Their chromaticity varies, and one star (GJ 3959) exhibits optical and NIR RV modulation consistent in both phase and amplitude. In general, though, we find that the NIR amplitudes are lower than their optical counterparts. We conclude that starspot modulation for rapidly rotating M stars frequently remains coherent for hundreds of stellar rotations and gives rise to Doppler signals that, due to this coherence, may be mistaken for exoplanets

TOI-2015b: A Warm Neptune with Transit Timing Variations Orbiting an Active mid M Dwarf

We report the discovery of a close-in ($P_{\mathrm{orb}} = 3.349\:\mathrm{days}$) warm Neptune with clear transit timing variations (TTVs) orbiting the nearby ($d=47.3\:\mathrm{pc}$) active M4 star, TOI-2015. We characterize the planet's properties using TESS photometry, precise near-infrared radial velocities (RV) with the Habitable-zone Planet Finder (HP) Spectrograph, ground-based photometry, and high-contrast imaging. A joint photometry and RV fit yields a radius $R_p~=~3.37_{-0.20}^{+0.15} \:\mathrm{R_\oplus}$, mass $m_p~=~16.4_{-4.1}^{+4.1}\:\mathrm{M_\oplus}$, and density $\rho_p~=~2.32_{-0.37}^{+0.38} \:\mathrm{g cm^{-3}}$ for TOI-2015b, suggesting a likely volatile-rich planet. The young, active host star has a rotation period of $P_{\mathrm{rot}}~=~8.7 \pm~0.9~\mathrm{days}$ and associated rotation-based age estimate of $1.1~\pm~0.1\:\mathrm{Gyr}$. Though no other transiting planets are seen in the TESS data, the system shows clear TTVs of super period $P_{\mathrm{sup}}~\approx~430\:\mathrm{days}$ and amplitude $\sim$$100\:\mathrm{minutes}$. After considering multiple likely period ratio models, we show an outer planet candidate near a 2:1 resonance can explain the observed TTVs while offering a dynamically stable solution. However, other possible two-planet solutions -- including 3:2 and 4:3 resonance -- cannot be conclusively excluded without further observations. Assuming a 2:1 resonance in the joint TTV-RV modeling suggests a mass of $m_b~=~13.3_{-4.5}^{+4.7}\:\mathrm{M_\oplus}$ for TOI-2015b and $m_c~=~6.8_{-2.3}^{+3.5}\:\mathrm{M_\oplus}$ for the outer candidate. Additional transit and RV observations will be beneficial to explicitly identify the resonance and further characterize the properties of the system.Comment: 28 pages, 15 figures, 6 tables. As submitted to AAS Journal

Frequency drift in MR spectroscopy at 3T

Purpose: Heating of gradient coils and passive shim components is a common cause of instability in the B-0 field, especially when gradient intensive sequences are used. The aim of the study was to set a benchmark for typical drift encountered during MR spectroscopy (MRS) to assess the need for real-time field-frequency locking on MRI scanners by comparing field drift data from a large number of sites.Method: A standardized protocol was developed for 80 participating sites using 99 3T MR scanners from 3 major vendors. Phantom water signals were acquired before and after an EPI sequence. The protocol consisted of: minimal preparatory imaging; a short pre-fMRI PRESS; a ten-minute fMRI acquisition; and a long post-fMRI PRESS acquisition. Both pre- and post-fMRI PRESS were non-water suppressed. Real-time frequency stabilization/adjustment was switched off when appropriate. Sixty scanners repeated the protocol for a second dataset. In addition, a three-hour post-fMRI MRS acquisition was performed at one site to observe change of gradient temperature and drift rate. Spectral analysis was performed using MATLAB. Frequency drift in pre-fMRI PRESS data were compared with the first 5:20 minutes and the full 30:00 minutes of data after fMRI. Median (interquartile range) drifts were measured and showed in violin plot. Paired t-tests were performed to compare frequency drift pre- and post-fMRI. A simulated in vivo spectrum was generated using FID-A to visualize the effect of the observed frequency drifts. The simulated spectrum was convolved with the frequency trace for the most extreme cases. Impacts of frequency drifts on NAA and GABA were also simulated as a function of linear drift. Data from the repeated protocol were compared with the corresponding first dataset using Pearson's and intraclass correlation coefficients (ICC).Results: Of the data collected from 99 scanners, 4 were excluded due to various reasons. Thus, data from 95 scanners were ultimately analyzed. For the first 5:20 min (64 transients), median (interquartile range) drift was 0.44 (1.29) Hz before fMRI and 0.83 (1.29) Hz after. This increased to 3.15 (4.02) Hz for the full 30 min (360 transients) run. Average drift rates were 0.29 Hz/min before fMRI and 0.43 Hz/min after. Paired t-tests indicated that drift increased after fMRI, as expected (p &lt; 0.05). Simulated spectra convolved with the frequency drift showed that the intensity of the NAA singlet was reduced by up to 26%, 44 % and 18% for GE, Philips and Siemens scanners after fMRI, respectively. ICCs indicated good agreement between datasets acquired on separate days. The single site long acquisition showed drift rate was reduced to 0.03 Hz/min approximately three hours after fMRI.Discussion: This study analyzed frequency drift data from 95 3T MRI scanners. Median levels of drift were relatively low (5-min average under 1 Hz), but the most extreme cases suffered from higher levels of drift. The extent of drift varied across scanners which both linear and nonlinear drifts were observed.</p

Procalcitonin Is Not a Reliable Biomarker of Bacterial Coinfection in People With Coronavirus Disease 2019 Undergoing Microbiological Investigation at the Time of Hospital Admission

Abstract Admission procalcitonin measurements and microbiology results were available for 1040 hospitalized adults with coronavirus disease 2019 (from 48 902 included in the International Severe Acute Respiratory and Emerging Infections Consortium World Health Organization Clinical Characterisation Protocol UK study). Although procalcitonin was higher in bacterial coinfection, this was neither clinically significant (median [IQR], 0.33 [0.11–1.70] ng/mL vs 0.24 [0.10–0.90] ng/mL) nor diagnostically useful (area under the receiver operating characteristic curve, 0.56 [95% confidence interval, .51–.60]).</jats:p

The development and validation of a scoring tool to predict the operative duration of elective laparoscopic cholecystectomy

Background: The ability to accurately predict operative duration has the potential to optimise theatre efficiency and utilisation, thus reducing costs and increasing staff and patient satisfaction. With laparoscopic cholecystectomy being one of the most commonly performed procedures worldwide, a tool to predict operative duration could be extremely beneficial to healthcare organisations. Methods: Data collected from the CholeS study on patients undergoing cholecystectomy in UK and Irish hospitals between 04/2014 and 05/2014 were used to study operative duration. A multivariable binary logistic regression model was produced in order to identify significant independent predictors of long (> 90 min) operations. The resulting model was converted to a risk score, which was subsequently validated on second cohort of patients using ROC curves. Results: After exclusions, data were available for 7227 patients in the derivation (CholeS) cohort. The median operative duration was 60 min (interquartile range 45–85), with 17.7% of operations lasting longer than 90 min. Ten factors were found to be significant independent predictors of operative durations > 90 min, including ASA, age, previous surgical admissions, BMI, gallbladder wall thickness and CBD diameter. A risk score was then produced from these factors, and applied to a cohort of 2405 patients from a tertiary centre for external validation. This returned an area under the ROC curve of 0.708 (SE = 0.013, p  90 min increasing more than eightfold from 5.1 to 41.8% in the extremes of the score. Conclusion: The scoring tool produced in this study was found to be significantly predictive of long operative durations on validation in an external cohort. As such, the tool may have the potential to enable organisations to better organise theatre lists and deliver greater efficiencies in care

Implementation of corticosteroids in treating COVID-19 in the ISARIC WHO Clinical Characterisation Protocol UK:prospective observational cohort study

BACKGROUND: Dexamethasone was the first intervention proven to reduce mortality in patients with COVID-19 being treated in hospital. We aimed to evaluate the adoption of corticosteroids in the treatment of COVID-19 in the UK after the RECOVERY trial publication on June 16, 2020, and to identify discrepancies in care. METHODS: We did an audit of clinical implementation of corticosteroids in a prospective, observational, cohort study in 237 UK acute care hospitals between March 16, 2020, and April 14, 2021, restricted to patients aged 18 years or older with proven or high likelihood of COVID-19, who received supplementary oxygen. The primary outcome was administration of dexamethasone, prednisolone, hydrocortisone, or methylprednisolone. This study is registered with ISRCTN, ISRCTN66726260. FINDINGS: Between June 17, 2020, and April 14, 2021, 47 795 (75·2%) of 63 525 of patients on supplementary oxygen received corticosteroids, higher among patients requiring critical care than in those who received ward care (11 185 [86·6%] of 12 909 vs 36 415 [72·4%] of 50 278). Patients 50 years or older were significantly less likely to receive corticosteroids than those younger than 50 years (adjusted odds ratio 0·79 [95% CI 0·70–0·89], p=0·0001, for 70–79 years; 0·52 [0·46–0·58], p80 years), independent of patient demographics and illness severity. 84 (54·2%) of 155 pregnant women received corticosteroids. Rates of corticosteroid administration increased from 27·5% in the week before June 16, 2020, to 75–80% in January, 2021. INTERPRETATION: Implementation of corticosteroids into clinical practice in the UK for patients with COVID-19 has been successful, but not universal. Patients older than 70 years, independent of illness severity, chronic neurological disease, and dementia, were less likely to receive corticosteroids than those who were younger, as were pregnant women. This could reflect appropriate clinical decision making, but the possibility of inequitable access to life-saving care should be considered. FUNDING: UK National Institute for Health Research and UK Medical Research Council

Non-steroidal anti-inflammatory drug use and outcomes of COVID-19 in the ISARIC Clinical Characterisation Protocol UK cohort: a matched, prospective cohort study.

Background: Early in the pandemic it was suggested that pre-existing use of non-steroidal anti-inflammatory drugs (NSAIDs) could lead to increased disease severity in patients with COVID-19. NSAIDs are an important analgesic, particularly in those with rheumatological disease, and are widely available to the general public without prescription. Evidence from community studies, administrative data, and small studies of hospitalised patients suggest NSAIDs are not associated with poorer COVID-19 outcomes. We aimed to characterise the safety of NSAIDs and identify whether pre-existing NSAID use was associated with increased severity of COVID-19 disease. Methods: This prospective, multicentre cohort study included patients of any age admitted to hospital with a confirmed or highly suspected SARS-CoV-2 infection leading to COVID-19 between Jan 17 and Aug 10, 2020. The primary outcome was in-hospital mortality, and secondary outcomes were disease severity at presentation, admission to critical care, receipt of invasive ventilation, receipt of non-invasive ventilation, use of supplementary oxygen, and acute kidney injury. NSAID use was required to be within the 2 weeks before hospital admission. We used logistic regression to estimate the effects of NSAIDs and adjust for confounding variables. We used propensity score matching to further estimate effects of NSAIDS while accounting for covariate differences in populations. Results: Between Jan 17 and Aug 10, 2020, we enrolled 78 674 patients across 255 health-care facilities in England, Scotland, and Wales. 72 179 patients had death outcomes available for matching; 40 406 (56·2%) of 71 915 were men, 31 509 (43·8%) were women. In this cohort, 4211 (5·8%) patients were recorded as taking systemic NSAIDs before admission to hospital. Following propensity score matching, balanced groups of NSAIDs users and NSAIDs non-users were obtained (4205 patients in each group). At hospital admission, we observed no significant differences in severity between exposure groups. After adjusting for explanatory variables, NSAID use was not associated with worse in-hospital mortality (matched OR 0·95, 95% CI 0·84–1·07; p=0·35), critical care admission (1·01, 0·87–1·17; p=0·89), requirement for invasive ventilation (0·96, 0·80–1·17; p=0·69), requirement for non-invasive ventilation (1·12, 0·96–1·32; p=0·14), requirement for oxygen (1·00, 0·89–1·12; p=0·97), or occurrence of acute kidney injury (1·08, 0·92–1·26; p=0·33). Interpretation: NSAID use is not associated with higher mortality or increased severity of COVID-19. Policy makers should consider reviewing issued advice around NSAID prescribing and COVID-19 severity. Funding: National Institute for Health Research and Medical Research Council